Repository navigation
Fix program lifetime and .ptd loading bugs in Module - #25412
Open
shoumikhin wants to merge 4 commits into
Open
shoumikhin wants to merge 4 commits into
shoumikhin wants to merge 4 commits into
Conversation
Module::program() returns a shared program, and its documentation says the data loader the program reads from stays valid for the lifetime of the program. But the loader belonged to the Module. Once the Module was destroyed, a program still held elsewhere kept a pointer to the deleted loader and used it whenever it loaded a segment on demand, for example when a later load_method() read the delegate data of a method from the file. That is a use after free. The program returned by program() now owns its loader. The loader is released together with the program, also when weak pointers to the program remain. Modules that are destroyed together with their program behave as before.
BundledModule::from_file reads the whole file into memory and gives the Module a loader that only borrows those bytes. The module frees them in its destructor. Module::program() hands out a shared pointer to the Program, and that Program keeps reading the same bytes. So a caller that keeps the Program after the BundledModule is gone reads freed memory. from_file now keeps the file data in a shared pointer, and gives the Module a SharedPtrDataLoader that shares it. The data lives as long as either the module or the Program needs it. The public constructor that takes a caller's buffer is unchanged: the caller still owns that buffer. The CMake build now also runs the bundled module test when devtools are built.
Module(std::shared_ptr<Program>, ..., data_map_loader) stores the loader of the program's external weights but never reads it. load() returns early once a program is set, so the .ptd data is never loaded, and every method that needs it fails to load. A program shared between Modules could therefore not use external weights at all. load() now loads the data maps of such a Module too. The data map code moves into one helper, which both paths call and which does nothing once the data maps are loaded. That also stops a load() retried after a failed Program::load from adding the .ptd data a second time and then failing with a duplicate key error instead of the program error. For such a Module, a bad .ptd now fails, with the .ptd error: load(), num_methods(), method_names(), method_meta(), format(), method(), get_mem_planned_buffer_sizes(), load_method(), and the calls built on load_method(), such as load_forward(), execute() and forward(). This includes methods that need no external data. is_loaded() is already true, because the program itself is loaded. Modules that load their own program are unchanged. This builds on the change that keeps a Module's data loader alive as long as its program.
Module::load opens the .ptd files it was given and adds each one to its list of data map loaders as soon as it is open. When opening a later file fails, or loading one of the data maps fails, the earlier entries stay. A retried load() then adds them a second time, and merging the data maps fails with a duplicate key error, even once the missing file exists. load() now opens every file before it keeps any, forgets the file names once the files are kept, and rebuilds its list of data maps on each attempt. So a retry adds each file once. Successful first loads are unchanged. This builds on the change that moves the data map code into one helper.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25412
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 1 PendingAs of commit 2e7a4b9 with merge base ba0d826 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is wrong today
Module::program()returns a sharedProgram, but the data loader that program reads from belongs to theModule. When theModuleis destroyed first, the program keeps a pointer to a deleted loader. The next segment it loads on demand, such as delegate data read by a laterload_method(), is a use after free.BundledModule::from_filefrees the file bytes in its destructor, but aProgramkept fromprogram()still reads them.Modulemade from a sharedProgramand adata_map_loadernever loads that data map, so methods that need the.ptdweights fail to load.load()fails on a missing.ptdfile, a bad data map, or a bad program, a retriedload()adds the earlier.ptddata a second time and fails with a duplicate key error.What this changes
There is one commit per fix, in this order:
program()now owns its loader. It is released with the last strong owner of the program.from_filekeeps the file bytes in a shared pointer that theModule's loader shares. The constructor that takes a caller's buffer is unchanged.bundled_module.hkeeps the same public API. The bundled module test is now also registered in CMake when devtools are built.Modulemade from a shared program, a bad.ptdnow failsload()and the calls that depend on it, such asmethod_meta(),num_methods(),load_method()andexecute(), also for methods that need no external data. The helper also stops a retry after a bad program from adding the.ptddata again.load()opens every.ptdfile before it keeps any, and rebuilds its data maps on each try, so a retry adds each file once.What was tested
TestProgramKeepsItsLoaderAlive,TestSharedProgramLoadsItsDataMap,TestLoadRetriesAfterAFailedProgramLoadandTestLoadRetriesAfterAMissingDataFile.TestProgramOutlivesModuleFromFilecan pass by luck, because it reads freed memory, so it was run under Valgrind: reads of freed memory without the fix, none with it. The one error left,Invalid alignment value: 0, comes from a moved-fromFileDataLoader. That is a separate bug, fixed in a separate pull request.